Learning CPG-based Biped Locomotion with a Policy Gradient Method: Application to a Humanoid Robot

In this paper we describe a learning framework for a central pattern generator (CPG)-based biped locomotion controller using a policy gradient method. Our goals in this study are to achieve CPG-based biped walking with a 3D hardware humanoid and to develop an efficient learning algorithm with CPG by reducing the dimensionality of the state space used for learning. We demonstrate that an appropriate feedback controller can be acquired within a few thousand The International Journal of Robotics Research Vol. 27, No. 2, February 2008, pp. 213–228 DOI: 10.1177/0278364907084980 c SAGE Publications 2008 Los Angeles, London, New Delhi and Singapore trials by numerical simulations and the controller obtained in numerical simulation achieves stable walking with a physical robot in the real world. Numerical simulations and hardware experiments evaluate the walking velocity and stability. The results suggest that the learning algorithm is capable of adapting to environmental changes. Furthermore, we present an online learning scheme with an initial policy for a hardware robot to improve the controller within 200 iter-